Connected ecosystem for laboratory environment

ABSTRACT

A connected ecosystem for a laboratory environment comprises an electronic lab notebook, and instrumented biosafety cabinet, and one or more sensing vessels containing cell cultures. The electronic lab notebook interfaces with the instrumented biosafety cabinet to provide instructions, guidance, and monitoring of a user during the set up of the experimental protocol and to receive commands from the user via one of several input modalities. After the experimental protocol has been set up in the instrumented biosafety cabinet, cell cultures may be moved to an incubator where the connected ecosystem may provide automatic monitoring of the cultures. The automatic monitoring is provided by sensors integrated into cell culture vessels and supplemented by images of the cell cultures captured by a camera. The user may be informed of deviations from expected results detected based on the automatic monitoring.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of priority under 35 U.S.C § 120 ofU.S. Provisional Application Ser. No. 62/941,001 filed on Nov. 27, 2019,and U.S. Provisional Application Ser. No. 62/940,991 filed on Nov. 27,2019, the contents of which are relied upon and incorporated herein byreference in their entireties.

FIELD OF THE INVENTION

This invention pertains to the field of laboratory work, and, inparticular, to the equipment used in a laboratory environment to protectboth laboratory workers and the samples and other materials used toconduct experiments in the laboratory environment.

BACKGROUND OF THE INVENTION

Modern laboratories use a variety of containment devices designed toprovide protection from contamination for both laboratory workers andmaterials and samples to conduct experiments. Experiments are oftenconducted in a biosafety cabinet (BSC). BSC's are enclosed, ventilatedhoods or workspaces that allow for the safe handling of pathogens,contaminants and other potentially hazardous materials. The primarypurpose of the BSC is to protect the samples of materials used in theexperiments conducted therein from outside contamination due to handlingby laboratory workers. In addition, BSC's also protect the laboratoryworker and the surrounding environment from biological contaminants andother hazardous materials.

There are currently three classes of BSC's. A class I BSC providesprotection to laboratory personnel and the laboratory environment butdoes not provide protection to samples or other materials containedwithin the BSC and used in the conducting of the experiments. Class IBSC's often have HEPA filtration systems that filter air exiting fromthe unit. Class II BSCs provide protection for both laboratory personneland samples and materials used in conducting the experiments. Class IIBSC's, are often provided with a front grill providing protection toboth the laboratory worker and the materials inside the BSC. Inaddition, a downward flow of HEPA-filtered air provides productprotection by minimizing the chance of cross-contamination across thework surface of the BSC. Class II BSC may also be provided with a HEPAfiltration system to filter air exiting from the unit. Class III BSEsare gas-tight enclosures with non-opening windows to allow viewing ofthe contents of the BSC and are often equipped with gloves attached tothe unit to allow access to materials. Class III BSC's are primarily forwork dealing with highly infectious microbiological agents and provide amaximum protection for both the worker and the environment.

Due to sterility requirements, laboratory workers are often cut off fromtheir most valuable tools, for example, computers and the protocols anddata that reside on them, when conducting experiments in the confines ofa BSC. In addition, laboratory workers may often wish to take notesregarding the experiments being conducted within the BSC. To accomplishthis, it may be necessary for the laboratory worker to remove theirhands from the interior of the BSC to access external tools. Dependentupon the sensitivity of the experiments being conducted within, workersmay need to remove gloves to access external tools and may then berequired to re-glove to reintroduce their hands to the interior of theBSC. Such movement of the hands back and forth between the laboratoryenvironment and the inside of the BSC may lead to contamination of thesamples and materials used in the experiments and may risk contaminationof the laboratory environment.

Additionally, cultures set-up within the confines of the BSC may oftenbe moved to an incubator for monitoring over a period of time. It mayoften be necessary to disturb the cultures to measure cell growth withinthe cultures or to obtain measurements of other parameters of theexperiments. For example, laboratory workers may wish to monitor therate of growth of the cultures, the confluence of the cultures, the pHof the cultures, the quantities and quality of materials present in thecell culture vessels, including, for example, oxygen or glucose, and thesterility of the environment.

To gather this data, it may be necessary to open the vessel to accessthe culture, thereby introducing the possibility of contamination of theculture. In addition, cell cultures may need to be monitored on anhourly or daily basis, requiring human intervention each time monitoringis required.

SUMMARY OF THE INVENTION

Embodiments of the invention comprise three major, interconnectedcomponents which together create a connected ecosystem for thelaboratory environment. These components include an electronic labnotebook, an enhanced biosafety cabinet, and sensing cell culturevessels.

The electronic lab notebook (ELN) is a combination of online datastorage and software which is operative to both store data representingthe results of laboratory experiments and as a means of conducting andmonitoring the protocols used in the experiments as well as the progressof cell cultures used in experiments. A laboratory worker may enter aprotocol for an experiment into the ELN. The protocol may include aninventory of required materials and objects, the steps required toset-up the protocol, and the parameters of expected results. The ELN maythereafter guide the laboratory worker through the protocol viastep-by-step instructions presented to the laboratory worker via aninterface with the BSC. The ELN is capable of monitoring materials andtheir quantities which are introduced into the interior of the BSC andchecking the materials and objects against the inventory entered by theuser as part of the protocol. After the experiment is set up in the BSC,the ELN is capable of interfacing with a plurality of sensing culturevessels to gather data on the progress of the cultures. The ELN iscapable of monitoring the metabolic state of cells in the cultures andmay detect signs of contamination. The laboratory worker may set upalarms or alerts within the protocol which will automatically alert thelaboratory worker if certain conditions are sensed within the culturevessels, for example, inflection points or milestones and data gatheredfrom the cell cultures. All data gathered from the cell cultures isstored in the data store for the experiment. The ELN may reside on andbe executed by a central computing system and preferably will have awireless interface with both the BSC and the sensing cell culturevessels.

The instrumented BSC, in addition to the basic components of anun-instrumented BSC, will contain enhancements allowing the laboratoryworker to interface with the ELN, for example, the BSC may be providedwith a surface on which the laboratory protocol and other informationmay be projected for viewing by the laboratory worker. In addition, theBSC may provide a voice recognition component that allows the laboratoryworker to enter notes into the ELN by voice, without the need to removethe hands from the interior of the BSC. The BSC will track consumablesbeing used in the experiment and will provide guidance and direction onsteps in the protocol based on the protocol entered into the ELN. TheBSC may further be equipped with a camera for stills or video which willrecord the conducting of the experiment or the setup of the cellcultures and may allow the automatic detection of various actions takenduring the setup of the experiment.

The last major component of the connected laboratory ecosystem consistsof sensing cell culture vessels. The vessels are equipped with varioussensors capable of monitoring the contents of the vessels, for example,oxygen, glucose or lactate sensors and/or a pH sensor, and conveyingthat information to the ELN. The ELN may thereafter monitor the inflowof data to determine inflection points or milestones indicated by thedata. The sensing cell culture vessels eliminate the need for constanthuman monitoring of the cultures and reduce the risk of contamination ofthe cell cultures which may be introduced by manual monitoring. Inaddition, the sensing vessels allow for continuous monitoring of thecultures instead of periodic monitoring by a laboratory worker.

According to a first aspect, a connected ecosystem for a laboratoryenvironment comprises a server hosting electronic lab notebook andprotocol data storage; and an instrumented biosafety cabinet incommunication with the electronic lab notebook.

A second aspect includes the system of the first aspect, wherein theelectronic lab notebook comprises a protocol input component forreceiving, from a user, a protocol definition defining an experimentalprotocol; a protocol set-up component for guiding the user throughset-up of the experimental protocol; a protocol monitoring component formonitoring progress of the experimental protocol; and a protocol datastorage for storing the defined experimental protocols and associateddata collected by the protocol monitoring component.

A third aspect includes the system of either the first or second aspect,wherein the protocol setup component provides step-by-step instructionsfor setup of the protocol to the user at the instrumented biosafetycabinet.

A fourth aspect includes the system of any of the first through thirdaspects, wherein the instrumented biosafety cabinet further comprises avideo output device for displaying video and still images on a surfaceof the instrumented biosafety cabinet; the step-by-step instructions forset-up of the protocol are displayed on the surface of the instrumentedbiosafety cabinet by the video output device.

A fifth aspect includes the system of any of the first through fourthaspects, wherein the instrumented biosafety cabinet further comprisesone or more scanners for scanning objects inserted into or extractedfrom the instrumented biosafety cabinet; and the electronic lab notebookmaintains a list of objects required for the set-up of the experimentalprotocol and an inventory of the objects inserted into or extracted fromthe instrumented biosafety cabinet and comparing the list with theinventory to determine if all objects required for the set-up of theexperimental protocol are present in the instrumented biosafety cabinet.

A sixth aspect includes the system of any of the first through the fifthaspects, wherein the instrumented biosafety cabinet further comprises avideo input device for capturing video and still images; and theelectronic lab notebook storing the video and still images in theprotocol data storage associated with the experimental protocol.

A seventh aspect includes the sixth aspect, wherein the electronic labnotebook analyzes the captured video to detect actions of the user indetermining when the actions of the user indicate that a step in theexperimental protocol has been completed.

An eighth aspect includes the system of either the sixth or seventhaspect, wherein the electronic lab notebook analyzing the captured videoto detect hand or eye gestures of the user and interpreting the hand oreye gestures as commands.

A ninth aspect includes the system of any of the first through theeighth aspects, the electronic lab notebook causing the instrumentedbiosafety cabinet to display, on a surface of the instrumented biosafetycabinet, one or more virtual buttons, the instrumented biosafety cabinetdetecting a user selection of one of the virtual buttons andinterpreting the user selection as a command.

A tenth aspect includes the system of any of the first through the ninthaspects, further comprising an audio output device for providing audiofeedback to the user regarding various aspects of the protocol set-up,the audio output device being integrated into the instrumented biosafetycabinet or worn by the user.

An eleventh aspect includes the system of any of the first through thetenth aspects, further comprising an audio input device, the audio inputdevice being integrated into the instrumented biosafety cabinet or wornby the user.

A twelfth aspect includes the system of the eleventh aspect, whereinaudio spoken by the user and collected by the audio input device isinterpreted as commands by the electronic lab notebook.

A thirteenth aspect includes the system of either the eleventh ortwelfth aspect, the audio spoken by the user includes a keyword spokenbefore the collected audio is interpreted as a command.

A fourteenth aspect includes the system of any of the first through thethirteenth aspects, further comprising one or more machine learningmodels trained to detect one or more of commands in audio spoken by theuser, hand or eye gestures of the user, objects inserted into areextracted from the instrumented biosafety cabinet or actions of the userto set-up the experimental protocol.

A fifteenth aspect includes the system of any of the first through thefourteenth aspects, wherein the protocol monitoring component receivesdata from one or more sensing vessels.

A sixteenth aspect includes the system of the fifteenth aspect, whereinthe data received from the one or more sensing vessels includingreadings from one or more sensors integrated with the one or moresensing vessels to provide measurements of various substances and thesensing vessels or environmental conditions of the sensing vessels.

A seventeenth aspect includes the system of either the fifteenth orsixteenth aspect, the data received from the one or more sensing vesselsincludes images of cell cultures contained in the one or more sensingvessels.

An eighteenth aspect includes the system of any of the fifteenth throughthe seventeenth aspects, the protocol monitoring component analyzing thedata received from the one or more sensing vessels to determine that oneor more milestones defined in the experimental protocol have beenachieved, or that one or more cell cultures in the one or more sensingvessels has deviated from expected results defined in the experimentalprotocol.

A nineteenth aspect includes the system of the eighteenth aspect, theprotocol monitoring component providing the user with a notification oralarm based on the analysis of the data received.

A twentieth aspect includes the system of either the nineteenth aspect,the notification or alarm being provided to the user on a personalcomputing device.

According to a twenty-first aspect, a biosafety cabinet comprises one ormore output devices for providing information to a user of the biosafetycabinet regarding setup of experimental protocols within the biosafetycabinet, the experimental protocols being stored in an electronic labnotebook associated with the experimental protocol; and one or moreinput devices for providing input data from the user regarding setup ofthe experimental protocol, the input data to be stored in the electroniclab notebook associated with the experimental protocol.

A twenty-second aspect includes the biosafety cabinet of thetwenty-first aspect, further comprising a video output device foroutputting videos and still images to be displayed on a surface of thebiosafety cabinet; an audio output device for outputting audio; a camerafor capturing videos or still images; one or more scanners; and an audioinput device for capturing audio.

A twenty-third aspect includes the biosafety cabinet of thetwenty-second aspect, wherein the videos and still images comprisingstep-by-step instructions guiding the user through the setup of theexperimental protocol.

A twenty-fourth aspect includes the biosafety cabinet of thetwenty-second or twenty-third aspect, the videos and still imagescomprising warnings to the user that the user has deviated fromparameters of the setup of the experimental protocol.

A twenty-fifth aspect includes the biosafety cabinet of thetwenty-second aspect, wherein the scanner tracks objects inserted intoand extracted from the biosafety cabinet, videos and still imagescomprising an inventory of objects required for the setup of theexperimental protocol and an acknowledgment that the required objectsfor setup of the experimental protocol are present in the biosafetycabinet.

A twenty-sixth aspect includes the biosafety cabinet of thetwenty-second through twenty-fifth aspects, the videos and still imagescomprising one or more virtual buttons displayed on the surface of thebiosafety cabinet, the biosafety cabinet being able to determine thatthe user has touched the area on the surface of the biosafety cabinet onwhich one of the virtual buttons is displayed.

A twenty-seventh aspect includes the biosafety cabinet of thetwenty-second through twenty-sixth aspects, the camera capturing imagesof objects in the biosafety cabinet and actions taken by the user duringsetup of the experimental protocol, the images being stored in theelectronic lab notebook associated with the experimental protocol.

A twenty-eighth aspect includes the biosafety cabinet of thetwenty-second through twenty-seventh aspects, the audio input devicecapturing commands spoken by the user.

A twenty-ninth aspect includes the biosafety cabinet of the twenty-firstthrough twenty-eighth aspects, further comprising one or more user inputdevices, the one or more user input devices being selected from a groupcomprising a touchscreen display, a mouse and a foot actuated switch.

A thirtieth aspect includes the biosafety cabinet of the twenty-secondaspect, further comprising: a processor; a network connection to aserver comprising electronic lab book logic and an experimental protocoldata storage containing one or more electronic lab books, eachelectronic lab book defining an experimental protocol including a listof materials and steps required to set-up the experimental protocol; andsoftware, for execution on the processor, the software configured tointerface, via the network connection, with the electronic lab booklogic to provide the functions of: receiving video, including stillimages, from the electronic lab book logic and displaying the video onthe surface of the biosafety cabinet; receiving audio from theelectronic lab book logic and playing the audio via the audio outputdevice; and receiving commands from the user and sending the commands tothe electronic lab book logic.

A thirty-first aspect includes the biosafety cabinet of the thirtiethaspect, the received video comprising step-by-step instructions for theuser to set-up the experimental protocol.

A thirty-second aspect includes the biosafety cabinet of the thirtiethor thirty-first aspect, the received video comprising an inventory ofobjects required for the setup of the experimental protocol indicatingwhich of the required objects have been inserted into the biosafetycabinet.

A thirty-third aspect includes the biosafety cabinet of any of thethirtieth through thirty-second aspects, the received video comprisingancillary information to assist the user in the setup of theexperimental protocol.

A thirty-fourth aspect includes the biosafety cabinet of any of thethirtieth through thirty-third aspects, the received video comprisingwarnings indicating the user has deviated from the step-by-stepinstructions for setting up the experimental protocol.

A thirty-fifth aspect includes the biosafety cabinet of any of thethirtieth through thirty-fourth aspects, the software comprising one ormore machine learning models trained to recognize one or more of voicecommands, hand or eye gestures of the user and specific users based onfacial recognition.

A thirty-sixth aspect includes the biosafety cabinet of the thirty-fifthaspect, the software further configured to provide the functions of:receiving voice input from the user via the audio input device;interpreting the received voice input as a command; and sending thecommand to the electronic lab notebook logic on the electronic labnotebook server via the network connection.

A thirty-seventh aspect includes the biosafety cabinet of thethirty-fifth or thirty-sixth aspect, the software further configured toprovide the functions of receiving video input from the user via thevideo input device, the video comprising hand or eye gestures of theuser; interpreting hand or eye gestures of the user in the receivedvideo as a command; and sending the command to the electronic labnotebook logic on the electronic lab notebook server via the networkconnection.

A thirty-eighth aspect includes the biosafety cabinet of any of thethirty-fifth through thirty-seventh aspects, the software furtherconfigured to provide the functions of receiving, via the video inputdevice and image containing a facial image of the user; and identifyingand authenticating the user based facial image recognition of the user.

A thirty-ninth aspect includes the biosafety cabinet of any of thethirtieth through thirty-eighth aspects, the software further configuredto provide the functions of: receiving video from the video inputdevice; and sending the video to the electronic lab notebook logic forstoring in the electronic lab notebook associated with the experimentalprotocol.

A fortieth aspect includes the biosafety cabinet of any of the thirtieththrough thirty-ninth aspects, the software further configured to providethe functions of receiving audio from the audio input device, the audiocomprising a note spoken by the user; and sending the note to theelectronic lab book logic for storing in the electronic lab notebookassociated with the experimental protocol.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a logical block diagram of the connected ecosystem forlaboratory environments.

FIG. 2 is a logical block diagram of the hardware components of the ELNserver.

FIG. 3 is a logical block diagram of the ELN logic.

FIG. 4 is a logical block diagram of the protocol data store, showingmultiple experiments stored therein.

FIG. 5 is a logical block diagram of the hardware components of theinstrumented BSC.

FIG. 6 shows an illustration of an instrumented BSC showing anexperimental protocol being displayed, showing the various stepsrequired to set-up the experiment.

FIG. 7 shows an illustration of an instrumented BSC showing a warningbeing given to a user as a user executes one step set-up theexperimental protocol.

FIG. 8 shows an illustration of a user using the instrumented BSC toenter a note into the electronic lab notebook using a voice command.

FIG. 9 is an illustration of an instrumented BSC showing the inventoryfunction wherein objects placed into the BSC are automatically tracked.

FIG. 10 shows an illustration of an instrumented BSC showing a series ofvirtual buttons displayed on the surface of the instrumented BSC fromwhich user may choose by touching.

FIG. 11 is a logical block diagram of the hardware components of thesensing vessel and the associated sensing plate.

FIG. 12 is an illustration of the sensing vessel and sensing plate ofFIG. 11 .

FIG. 13 is an illustration of a user receiving a status message from anelectronic notebook as the cultures and experiment are continuouslymonitored for various results.

FIG. 14 is an illustration of a user receiving readings from sensingvessels, showing deviations in the actual measurements from the expectedresults.

FIG. 15 shows a computing architecture suitable for supporting thefunctions of the connected ecosystem for the laboratory environment asdescribed herein.

DEFINITIONS

The singular forms of the terms “a,” “an” and “the” include pluralreferents unless the context clearly dictates otherwise. The endpointsof all ranges reciting the same characteristic are independentlycombinable and inclusive of the recited endpoint. All references areincorporated herein by reference.

As used herein, the terms “have,” “having,” “include,” “including,”“comprise,” “comprising” or the like are used in their open-ended sense,and generally mean “including, but not limited to.”

As used herein, the terms “top”, “bottom”, “side”, “upper”, “lower”,“above”, “below” and the like are used herein for descriptive purposesand not necessarily for describing permanent relative positions. Itshould be understood that the terms so used are interchangeable underappropriate circumstances such that embodiments of the presentdisclosure are, for example, capable of operation in other orientationsthan those illustrated or otherwise described herein.

As used herein, the term “protocol” or “experimental protocol” is usedto refer to any activity taking place within the confines of aninstrumented BSC 500 and governed by and documented in an ELN stored inprotocol data store 400.

All scientific and technical terms used herein have meanings commonlyused in the art unless otherwise specified. Any definitions providedherein are to facilitate understanding of certain terms used frequentlyherein and are not meant to limit the scope of the present disclosure.

DETAILED DESCRIPTION

Reference will now be made in detail to the present embodiment(s), anexample(s) of which is/are illustrated in the accompanying drawings.Whenever possible, the same reference numerals will be used throughoutthe drawings to refer to the same or like parts.

The present disclosure is described below, at first generally, then indetail on the basis of several exemplary embodiments. The features shownin combination with one another in the individual exemplary embodimentsdo not all have to be realized. In particular, individual features mayalso be omitted or combined in some other way with other features shownof the same exemplary embodiment or else of other exemplary embodiments.

FIG. 1 is a high-level logical diagram of the main components of theconnected ecosystem 100 for a laboratory environment in accordance withvarious embodiments of this invention. Electronic Lab Notebook (ELN)server 200 lies at the heart of the system and provides a point ofinterconnection for all other components of the system. ELN server 200may interface with an instrumented BSC 500 and a plurality of sensingculture vessels 1100 used to carry out an experimental protocol. Itshould be realized that, although only a single instrumented BSC isshown, multiple instrumented BSC's in a laboratory environment may beconnected to ELN server 200 simultaneously. Likewise, sensing culturevessels 1100 from a plurality of different experimental protocols mayalso interface with ELN server 200 simultaneously.

ELN server may execute ELN logic 300. ELN logic 300 is responsible forguiding a user through the definition of an experimental protocol, theinitial set-up of an experimental protocol, including materials to beused in protocol and the parameters of the protocol, and monitoring ofthe protocol after the experiment has been set-up in the instrumentedBSC 500.

A user may interface with ELN logic 300 using a user device 150 accessand interface allowing the user to specify the parameters of theexperimental protocol, including, for example, the required equipmentand supplies, the required biological materials, the environmentparameters under which the experimental protocol should be set-up andmonitored, and the expected results of the growth of cultures after theinitial setup of the protocol.

ELN logic 300 may interface with instrumented BSC 500 to guide the userthrough the physical setup of cultures and to otherwise interface withthe user while the user is engaged with the experimental protocol at theinstrumented BSC 500. Lastly, ELN logic 300 may interface with aplurality of sensing culture vessels 1100, typically placed in anincubator environment after removal from instrumented BSC 500, tomonitor various parameters of the protocol and to determine if theprotocol conforms to the expected results expressed by parametersentered by the user.

ELN server 200 may be, for example, any type of computing device wellknown in the art capable of executing the ELN logic 300 and interfacingvia network 110 with instrumented BSC's 500 and sensing culture vessels1100. A typical architecture for implementing ELN server 200 may befound in FIG. 15 and will be discussed later. ELN server 200,instrumented BSC's 500 and sensing culture vessels 1100 may communicateover a network which may be any type of network now known or laterdeveloped, including, for example, wired connections, Wi-Fi, Bluetooth,near field, 4G or 5G cellular networks, etc. Preferably, the componentsof the connected ecosystem will implement security protocols, including,for example encryption schemes, to maintain the integrity and secrecy ofthe data exchange between the various components of the system.

Protocol data store 400 is used to store the various experimentalprotocols and, in addition, stores data collected from instrumentedBSC's 500 during set-up of the experiments and from sensing culturevessels 1100 to monitor culture growth and progress. As would berealized by one of skill in the art, data store 400, while shown asbeing part of ELN server 200, may reside anywhere within the network110, for example, as a cloud service (not shown), as part of anotherserver (not shown) or as part of another component within the connectedecosystem 100.

User device 150 may be used to interface with ELN logic 300, forexample, to set-up experimental protocols, to store notes during set-upof the experiments within the confines of the instrumented BSC 500 andto monitor progress of cell cultures within sensing culture vessels1100. In addition, user device 150 may be used to refer to video orstill pictures taken during set-up of the experiment instrumented BSC500, to refer to audio notes taken during set-up of the experiment andto view images of cultures in sensing culture vessels 1100. In addition,user device 150 may receive alerts from ELN logic 300 alerting the userto various milestones reached during the experimental protocol and toreceive alerts regarding experimental deviations from the expectedprotocol results. User device 150 may be, for example, a desktopcomputer, laptop computer, a computing tablet, smart phone, etc.

Electronic Lab Notebook (ELN)

FIG. 2 is a logical diagram of the hardware components of ELN server200. As previously discussed, ELN logic 300 resides on ELN server 200and is executed thereon. ELN logic 300 may be capable of handling aplurality of separate experimental protocols, and interfacing with theplurality of separate instrumented BSC's 500 and sensing vessels 1100,simultaneously. ELN logic 300 may interface with various hardwarecomponents present in both the instrumented BSC 500 and in sensingculture vessels 1100. In addition, ELN logic 300 may provide variousoutputs, including, for example, video, sound and alarms.

Camera input 202 may take input from video or still cameras located ininstrumented BSC 500 or in proximity to sensing culture vessels 1100.Camera input 202 may be ephemeral in nature or may be permanently storedin protocol data store 400. ELN logic 300 may act passively on camerainput 202, for example, by storing it for later reference in protocoldata store 400. Alternatively, ELN logic 300 may use the camera input202 as active input to the protocol. For example, a video camera mayrecord the actions of a user in setting up an experiment within theconfines of an instrumented BSC 500. The ELN logic 300 may be able torecognize certain aspects of the experimental protocols from camerainput 202. For example, ELN logic may be able to recognize hand gesturesof the user and respond in various predefined ways. ELN logic mayfurther be able to recognize various objects physically inserted in orremoved from the instrumented BSC 500 and may be able to record thetimes and instances where these events occur. In addition, ELN logic 300may be able to use camera input 202 to monitor the user's actions duringset-up of the experimental protocol within instrumented BSC 500 and maybe able to determine if the user is following the defined protocol forthe experiment.

Cameras providing camera input 202 to ELN server 200 may be video orstill cameras physically located in the instrumented BSC 500 orexternal, but in close proximity to instrumented BSC 500 such as, acamera integrated into an object worn by the user, for example, aheadband or goggles having an integrated camera. The cameras providingcamera input 202 may communicate directly with ELN server 200 vianetwork 110 or may be integrated into a data stream generated byinstrumented BSC 500 and communicated to ELN server 200 via network 110.Users may have the ability to turn the camera on or off to start or stoprecording via voice command, hand gestures or via other types ofcontrols located at or near the instrumented BSC 500.

Camera input 202 may also comprise a plurality of video or still imagestreams from sensing vessels 1100, which may be used to determine if thecultures being monitored within sensing vessels 1100 are following theexpected results of the protocol or are deviating from expected results.

Voice input 206 may take as input an audio stream generated by an audiotransducer at or near the instrumented BSC 500. The audio transducer maybe integrated into an object worn by the user, for example, a headband,goggles, or a headset. ELN logic 300 may have the ability to providevoice-to-text translation or may utilize a natural language processortrained by machine learning to determine the intent of voice inputsprovided by a user. A user may be able to provide commands to ELN logic300 via voice commands received via voice input 206. For example, theuser may be able to instruct ELN logic 300 to “start video recording” orto “turn on BSC illumination”, etc. In addition, the user may utilizevoice input 206 to take notes during set-up of the experimental protocolin the instrumented BSC 500. Such notes may be translated by the naturallanguage processor to text and stored as text or may be stored as audiosnippets in protocol data store 400.

Audio transducers providing voice input 206 may be, for example,microphones which may transmit raw audio as voice input 206 or may be,for example, intelligent assistants which may be able to recognize voicecommands locally and inform ELN logic 300 of the command spoken by theuser. ELN logic 300 may require an attention word or phrase prior toacting on voice input 206 to avoid having to interpret casualconversations of the user or other sounds generated in the vicinity ofthe instrumented BSC 500.

Scanner input 208 may be received from any type of scanner locatedwithin or in close proximity to the instrumented BSC 500. Examples ofsuch scanners may include a barcode scanner, a QR code scanner, an RFIDscanner, etc. Scanner input 208 may be used to monitor objects ormaterials introduced into or extracted from the instrumented BSC 500. Inaddition, scanner input 208 may be received from incubators into whichsensing culture vessels 1100 are placed to determine when cultures areintroduced into the incubator environment.

User input 210 may be received from other physical or virtual deviceslocated in the environment of the instrumented BSC 500. For example,instrumented BSC 500 may provide a mouse or physical keyboard for humaninput, or a virtual keyboard or other virtual buttons which may beprojected, using a video projector or a laser, on the internal surfaceof the instrumented BSC 500, wherein the instrumented BSC 500 may beable to detect when the user has pressed one of the virtual buttons. Forexample, the user may be provided with physical or virtual buttonsindicating that various steps of the protocol have been completed andmay indicate to the instrumented BSC 500 and to ELN logic 300 that astep in the protocol is been completed.

Camera input 202, voice input 206, scanner input 208 or user input 210may be used to identify a user engaged with an instrumented BSC 500. Forexample, a user may be recognized by facial recognition softwareintegrated with ELN logic 300 to recognize an image of a face inputtedas camera input 202, by voice recognition software operating on voiceinput 206, by identification received by sensing a barcode, QR code orRFID tag located on an identification badge of the user and received ascanner input 208 or via a password input as user input 210. ELN logic300 may implement authentication protocols in accordance with any of thementioned methods to authenticate the user prior to allowing access toan experimental protocol.

Sensors input 212 may receive input from sensors located in a pluralityof sensing culture vessels 1100 to monitor progress of cultures for theexperimental protocol. Information received via sensors input 212 may bestored in protocol data store 400 and may be used by ELN logic 300 todetermine progress of the experimental protocol or to detect deviationsfrom expected results.

ELN logic 300 may be operative to allow protocol definition, monitorprotocol set-up, and monitor progress of protocol over time. A user mayinterface with ELN logic via a user interface 240 accessed via userdevice 150. The user interface may be provided, for example, via a webbrowser, via client software resident on user device 150 which mayinterface directly with user interface 240 or ELN logic 300 via network110. Additionally, ELN server 200 may itself be provided with the userinterface 240 allowing direct user interaction with ELN logic 300.

User interface 240 may be used by the user to define an experimentalprotocol, as discussed below. In addition, the user interface 240 may beused to define parameters of expected results of cultures within sensingculture vessels 1100. Lastly, the user may use user interface 240 toset-up any alerts or milestones for which the user wishes to receivenotification and to actually receive notifications on user device 150.

ELN logic 300 may provide various outputs to the instrumented BSC 500and to user device 150 via user interface 240. ELN logic 300 maygenerate audio which may be sent as audio output 224 to be renderedwithin instrumented BSC 500 or to user device 150. For example, ELNlogic may provide audio cues for the user to guide the user in the setupof the experimental protocol within the instrumented BSC 500. As anexample, a user may indicate via user input 210 that he or she is readyto move to a next step of the protocol, upon which ELN logic 300 maycause specific instructions to be played as audio output 224. Audiooutput 224 may be rendered via a speaker located in instrumented BSC 500or may be sent to a user via a personal audio device, for example, aheadset, either directly or via user device 150. ELN logic 300 may alsocause other types of audio to be sent as audio output 224, for example,the user may wish to have music played during the setup of anexperiment.

ELN logic 300 may generate video output 222 to be rendered ininstrumented BSC 500. It should be noted that the term “video output” asused herein may refer to both video and still images. Video output 222may comprise, for example, a video or image showing the next steps inthe experimental protocol to be executed by the user. Such video output222 may also be accompanied by audio output 224. The video output 222may show which material should be used in the next step of the protocolby the user and the quantities of such materials and may indicate otherspecific steps be taken by the user during the setup of the experimentalprotocol. The video output may be stored in protocol data store 400, maybe taken from a library of stored video or still images used in otherprevious experiments or may be provided from other sources, for example,the Internet.

ELN logic 300 may be capable of monitoring experimental protocols anddetermining if the results of the protocols, based on sensors input 212received from sensing culture vessels 1100 conform to defined expectedresults for the protocol. ELN logic 300 may generate a notificationoutput 220 indicating, for example, that a particular milestone in theexperimental protocol has been reached or that data received as sensorinput 212 from sensing culture vessels 1100 indicates that certaincultures have deviated from expected results as set-up by the user viauser interface 240. In addition, notifications output 220 may be usedsimply to report the results of monitoring of cultures in sensingculture vessels 1100. Users may be notified of such alarms via userinterface 240 and may receive the alarms on the user device 150 as textmessages, emails, or via software resident on the client device specificto the system.

ELN logic 300 may also interface with other devices or interfaces 160.For example, third-party devices or devices yet to be developed forintegration with the connected ecosystem 100 defined herein and used inconducting an experimental protocol may be used with ELN logic 300.Examples of other devices may include, for example, smart pipettors andsmart incubators. In addition, ELN logic 300 may interface with othersystems, for example, external libraries of data, systems providingvarious services via APIs, email or other messaging systems, etc., whichmay be useful in conducting and documenting the experimental protocols.

FIG. 3 is a logical block diagram of the ELN logic 300. ELN logic 300 isoperative to oversee the entire experimental protocol, from setup toconclusion of the experiments. ELN logic 300 includes protocol inputcomponent 310 allowing users to define protocols, protocol setupcomponent 320 for guiding a user through the setup of an experimentalprotocol within an instrumented BSC 500 and protocol monitoringcomponent 330 for monitoring cell cultures via data collected fromsensing culture vessels 1100 and other sources. It should be realized byone of skill in the art that the breakdown of components of ELN logic300 is provided only as one possible embodiment. Different arrangementsof functional components may be provided in any configuration to providethe functions of ELN logic 300.

Protocol input component 310 receives a definition of the experimentalprotocol from a user via user interface 240, including, for example, alisting of materials and equipment needed for setting up of the protocol(e.g., culture plates or vessels, culture media, pipettors, pipettetips, etc.), various biological materials needed, the quantities of suchmaterials, the order of steps in the setting up the protocol,environmental conditions for setting up the protocol and any otherparameters or special instructions necessary to define the protocol. Theprotocol definition may also include other information, for example, howoften the sensors from sensing culture vessels 1100 need to be read, theexpected quantities of various materials produced or consumed by thecultures, for example, lactate or glucose, and other parameters of thecultures, for example, temperature, rate of growth, pH cell convergence,etc.

The protocols definition may further include any alerts or milestonesfor which the user wishes to receive notifications. For example, theuser may wish to be notified when certain defined milestones have beenreached as cultures are being grown or may wish to receive notificationsregarding deviations of certain cultures from expected results withrespect to certain parameters. Notifications can be provided via userinterface 240 directly to the user device 150 by any well-known means,for example, via text message, email, public notification, or viaspecific software resident on client device 150. Experimental protocols,after being defined by the user and received via protocol input 310, maybe stored in protocol data store 400.

Protocol setup component 320 handles activities undertaken by user toset-up the experimental protocol within the integrated BSC 500. Protocolsetup component 320 may guide the user through the setup of theexperimental protocol. The defined protocol stored in protocol datastore 400 and input via protocol input component 310 may be used toprovide step-by-step instructions for the user to set-up the culturesused in the experiment protocol. The protocol may be displayed invarying levels of detail in the instrumented BSC 500. The instructionsto the user may be provided by ELN logic 300 via video output 222 oraudio output 224 to either the instrumented BSC 500 or directly to auser device 150. Instrumented BSC 500 may comprise means for displayingvideo, for example, an integrated screen or projector for projectingvideo or images on a wall of the integrated BSC 500 and/or an integratedspeaker for playing audio. FIG. 6 shows a protocol being displayed on awall or glass hood of an instrumented BSC 500 having several steps,wherein the current step is shown highlighted in greater detail.Protocol setup component 320 may be able to track various stepsundertaken by the user and the setup of the experimental protocol viavarious means of feedback from the user, including, for example,explicit commands entered or spoken by the user or actions of the userrecognized from video. Protocol setup component may be able to notifythe user, based on the recognized action, when the user has deviatedfrom the predefined experimental protocol.

Protocol setup component 320 may receive inputs from the user regardingvarious actions taken by the user. These inputs may be determinedautomatically via camera input 202 or voice input 206 or may be madeexplicitly by the user through user input 210. The user may indicate tothe protocol setup component 320 that various actions have beenundertaken by the user, for example, the user may indicate that aparticular step of the protocol set-up has been completed and that theuser is ready to move on to the next step.

Protocol setup component 320 may receive inputs from other smart devicesused during the set-up of the protocol. For example, the protocol mayrequire the use of a “smart pipettor”, that can sense and regulate theamount of fluid dispensed from a pipettor into a plate or culture vesselwell. Based on the protocol, the instrumented BSC 500, integrated withthe pipettor, can regulate the amount of media, for example, injectedinto a well plate for cell growth or determining cell toxicity.

Protocol setup component 320 may include one or more machine learningmodels trained to detect various events or objects within video, stillimages or voice inputs received via a camera input 202 or voice input206. For example, protocol setup component 320 may include a machinelearning model trained to recognize human gestures in a video and may beused by ELN logic 300 to determine that the user has requested someaction occur via a hand or eye gesture. A machine learning model trainedto recognize human faces may be used to identify users carrying out theexperiments and may authenticate such users to maintain the integrityand secrecy of the protocols and the data produced by them. A machinelearning model trained to recognize certain voice commands may beutilized to receive voice input from the user to request that someaction occur. A machine learning model trained to recognize variousobjects may be used to identify objects inserted into or extracted fromthe instrumented BSC 500. Protocol setup component 320 may also beprovided with a natural language processor to process spoken languageand translate the spoken language to textual input for storing as notesduring the setup of the experimental protocol. Spoken language may alsobe stored as audio snippets within protocol data store 400.

Protocol setup component 320 may provide an augmented reality experiencefor a user of the instrumented BSC 500. The augmented reality experiencemay be an interactive experience of the real-world environment ofinstrumented BSC 500 where the objects that reside in the instrumentedBSC 500 are enhanced by computer-generated perceptual information, whichmay include multiple sensory modalities. Such modalities may include,for example, visual, auditory, haptic, somatosensory and olfactorymodalities. Instrumented BSC 500 may be equipped with transducersallowing provision of the various modalities of feedback.

Protocol monitoring component 330 handles determining the status of theexperimental protocol post set-up and reporting on the progress of cellcultures set-up within the instrumented BSC 500 in sensing culturevessels 1100, which are typically transferred to an incubator afterset-up in instrumented BSC 500. The defined protocol input via protocolinput component 310 may contain definitions or parameters indicatingexpected results of the culturing of cells, including, for example, thepresence or absence of various substances within sensing culture vessels1100 (e.g., glucose, lactate, dissolved oxygen), the rate of growth ofthe cell cultures, the cell confluence, cell morphology, pH, humidity,temperature etc. Sensing culture vessels 1100 may be provided with avideo or still camera and ELN logic 300 may receive data from thesensing culture vessels 1100 via a camera input 202 and may be capableof analyzing the received data and/or video or still images to determinethe progress of the cell cultures. Protocol monitoring component 330 mayemploy one or more machine learning models trained to recognize variousparameters of the cell cultures, for example, machine learning model maybe trained to recognize the rate of growth of cells. In addition,protocol monitoring component 330 may have the ability to analyze pastexperimental results to recognize data trends, either graphically or viaa machine learning model trained to recognize data trends. Notificationsto the user may be provided by the user interface 240 regardingmilestones of the experimental protocol, progress of the cell culturesand deviations of the cell cultures from the defined parameters ofprotocol.

FIG. 4 shows details of protocol data store 400. Protocol data store 400may store a plurality of experiments, shown in FIG. 4 as experiments 1 .. . N. Each experiment may store protocol definition 402, input by theuser via protocol input component 310 in ELN logic 300. As previouslydiscussed, protocol definition 402 may include an inventory of materialsrequired to set-up the experiment, biological materials required for theexperiment, steps in the protocol, quantities of materials to be used,environmental conditions of the setup of the protocol, and any otherinformation necessary for the user to successfully set-up theexperimental protocol. In addition, protocol definition 402 may includenotifications or alerts requested by the user and to be delivered to theuser via the user device 150, which may include, for example,notifications of milestones reached in the growth of cell cultures,changes in materials within the cell cultures and deviations of the cellcultures from expected results.

Protocol data store 404 stores data collected from sensing vessels 1100which may be used by ELN logic to assess the results of the experimentalprotocol against expected results set forth in protocol definition 402.In addition, protocol data store 404 may include data collected duringthe set-up of the protocol, for example, video or still images takenduring the setup, user notes entered during the setup, information fromother smart devices used during the setup, etc. ELN logic 300 may haveaccess to experiments stored within protocol data store 400 and protocolsetup component 320 may use the protocol definition 402 to guide theuser through the set-up of the protocol.

Instrumented Biosafety Cabinet (BSC)

Instrumented BSC 500 is an enhanced version of a standard class I, classII or class III biosafety cabinet that comprises components necessaryfor integration of instrumented BSC 500 with ELN logic 300. FIG. 5 showsa logical block diagram of the hardware components of instrumented BSC500. It should be noted that instrumented BSC 500 may comprise anexisting biosafety cabinet retrofitted with the enhancements necessaryfor integration with ELN logic 300 or may be a newly manufacturedbiosafety cabinet having the required components built in.

Instrumented BSC 500 may comprise one or more means to display videoand/or still images to a user. In preferred embodiments of theinvention, video output 510 may comprise a projector used to projectvideo or still images on a surface of the instrumented BSC, for example,the back wall or the front glass. In alternative embodiments, videooutput 510 may comprise a video display which may be a dedicated videodisplay or may be a video display of another device, for example, tabletcomputing device which has been temporarily brought into the environmentof the instrumented BSC 500. In yet other embodiments, video output 510may comprise a user wearable device, for example, goggles having thecapability of displaying video or an immersive headset suitable for usein displaying augmented reality experiences. Devices wearable by theuser or otherwise external to instrumented BSC 500 may be connected tothe instrumented BSC 500 via a wireless connection, for example Wi-Fi orBluetooth.

Video output 510 may be used to display any information contained withinprotocol data store 400 regarding the experimental protocol currentlybeing set-up within instrumented BSC 500. This may include, for example,displaying the overall protocol at varying levels of detail, which mayinclude displaying various steps of the protocol and the actions whichmust be undertaken by the user to implement the protocol, as shown inFIG. 6 . The various steps of the protocol may be displayed as anoverview, shown as reference number 602 in FIG. 6 wherein the currentstep may be provided in more detail as shown by reference number 604 inFIG. 6 . Video output 510 may also display other related information,for example, videos showing how the user is to implement various stepsof the protocol or information regarding objects and materials (e.g.,spec sheets, etc.) used in the setup of the protocol, shown as referencenumber 606 in FIG. 6 . As an example, FIG. 7 shows a user receivingwarning 702 indicating that that the user is filling a culture toquickly and may exceed the required quantity of materials in theculture. In short, any information essential for or peripheral to thesetup of the experimental protocol may be displayed using video output510.

In another aspect of the invention, the video output 510 of instrumentedBSC 500 may act in conjunction with camera 520 to provide magnifiedviews of objects within the instrumented BSC 500 on video output 510.

Instrumented BSC 500 may also comprise one or more means to displayaudio output. In one embodiment of the invention, audio output 512 mayinclude, for example, one or more speakers mounted on the interiorand/or exterior of the instrumented BSC 500. In other embodiments of theinvention, audio output 512 may include a headset wearable by the userand connected to the instrumented BSC 500 or to ELN logic 300 via awired or wireless connection, for example, Wi-Fi or Bluetooth.

Audio output 512 may act in conjunction with video output 510 to provideany audio accompanying displayed videos or still images. In addition,audio output 510 may be used to allow the user to hear narration oraudible instructions regarding actions necessary to the set-up of theexperimental protocol. Audio output 512 may also be used, for example,to play music or other entertainment for the benefit of the user whileworking with the instrumented BSC.

In another aspect of the invention, video output 510 and audio output512 may be configured to act as a mirror of the personal computingdevice of the user which may allow the user to engage in phone or videoconversations or otherwise interact with other applications on apersonal computing device while using the instrumented BSC 500.

Instrumented BSC 500 may also be equipped with other forms of output 514which may include, for example, transducers providing haptic,somatosensory and olfactory feedback to the user. These other forms ofoutput 514 may be used in conjunction with video output 510 and audiooutput 512 to provide the user with an augmented reality experiencewhile using instrumented BSC 500. The augmented reality experience may,for example, show augmented views of objects within instrumented BSC500.

Instrumented BSC 500 is also equipped with various forms of input foruse by the user in communicating with ELN logic 300. The various formsof input may be used to enter information which will become part of therecord stored in the protocol database 400 for the current experimentalprotocol or may be commands to ELN logic 300. Users may documentprotocol set-up by augmenting the ELN for the protocol with video orstill images or audio narration.

Camera 520 may include one or more video and/or still cameras, orcameras capable of collecting both video and still images and may beused for several purposes. In preferred embodiments, camera 520 may beused to document the setup of and experimental protocol by augmentingthe ELN for the experimental protocol with videos or still imagesshowing how the protocol was set-up by the user. In some embodiments,the video stream may be analyzed to determine the user's progress in thesetup of the experimental protocol, for example, by recognizing when theuser has performed a specific activity and/or may be used to recognizeobjects used in the setup of the protocol. Such recognition may beperformed by machine learning models trained to recognize specificactivity or objects.

In some embodiments of the invention, camera 520 may be used to providecommands to ELN logic 300 via various hand or eye gestures which may berecognized by ELN logic 300. Hand or eye gestures may be recognized invideo streams using machine learning models trained to recognize suchgestures. The gestures and their associated meanings can be pre-definedby ELN logic 300 or may be configured by individual users based onpersonal preferences.

In yet other embodiments of the invention, camera 520 may be used toperform identification of the user for security and data confidentialityreasons. Camera 520 may be configured to recognize faces within videostreams or still images and may use facial identification toauthenticate the user. Facial identification may be performed by machinelearning models trained to recognize specific individuals.

Instrumented BSC 500 is also equipped with one or more microphones 522to allow the user to provide audible instructions, feedback and/or notesto ELN logic 300. In one embodiment, microphone 522 may be internally orexternally integrated with instrumented BSC 500 or may be provided aspart of a headset worn by the user and connected via a wirelessconnection, for example, Wi-Fi or Bluetooth to instrumented BSC 500 orto ELN logic 300.

In some embodiments, microphone 522 may be used to provide commands toELN logic 300. For example, the user may instruct ELN logic 300 via aspoken command to “proceed to the next step of the protocol”. Suchcommands may be recognized by a natural language processor or by amachine learning model trained to recognize commands from spokenlanguage. In some embodiments, commands may be initiated by precedingthe command with an audible cue, such as a keyword or keywords, forexample, “Hey Hood . . . ”, as shown by reference number 802 in FIG. 8 ,to alert ELN logic 300 that the following spoken language comprises acommand. Audio commands recognized by ELN logic 300 may be displayed byvideo output 510 as shown in FIG. 8 .

In some embodiments microphone 522 may be used to enter audible notes tothe ELN, as shown in FIG. 8 , which may be stored in the ELN associatedwith the experimental protocol in protocol data store 400. The audiblenotes may be initiated by issuing a specific command to ELN logic 300indicating that the following spoken language is to be stored as a note,such as “Make a note” as shown in FIG. 8 . Stored notes may comprise anaudio snippet or may be processed by a natural language processor andconverted to text for storage.

In some embodiments, microphone 522 may also be used to allow the userto engage with applications on a personal computing device 150, forexample, to engage in a phone or video conversation or to interact withother applications on the personal computing device.

Instrumented BSC 500 may be equipped with one or more scanners 524 fortracking objects and consumables introduced into or extracted frominstrumented BSC 500. Such scanners 524 may comprise barcode scanners,QR scanners, RFID scanners or any type of other scanner now known orlater developed. In addition, cameras 520 may also be used to capturebarcodes, QR codes or any other visual code identifying objects andconsumables. The user may indicate via a voice command or by other meansthat an object is entering or exiting the instrumented BSC 500, and thenmay scan the object. The object may be identified based on the scan anda note may be entered into the ELN for the experimental protocolindicating the time the object was scanned and whether the object isentering or exiting integrated BSC 500. Video output 522 may display aninventory checklist of objects required for the set-up of theexperimental protocol as shown as reference number 902 in FIG. 9 . Asobjects are brought into the interior of instrumented BSC 500, scanners524 may recognize the objects and provide a check mark in the inventorychecklist 902 showing that the object has been provided. Alternatively,an object brought into the interior of instrumented BSC 500 may berecognized by analyzing video input provided by camera 520 using amachine learning model trained to recognize objects or by reading abarcode or QR code attached to the object. The tracking of objects andconsumables entering or exiting instrumented BSC 500 may be used by ELNlogic 500 for purposes of tracking of inventory of supplies andreordering of supplies when needed and for issuing warnings in the eventthat objects are inserted into instrumented BSC 500 that are notrequired for the protocol (i.e. do not appear on the inventory ofobjects and materials entered as part of the experiment a protocol).

In an additional embodiment, scanners may be used to aid in theidentification of the users conducting the experimental protocol byscanning an identification badge or token of the user such as toauthenticate the user to access the ELN for the experimental protocolstored in protocol data store 400.

Instrumented BSC 500 may be equipped with other forms of user inputdevices 526. Such other forms of user input devices 526 may include, forexample, a mouse, a touch sensitive screen, (e.g. of a tablet computingdevice), an electronic pencil, a foot pedal, and virtual buttonsdisplayed on surfaces of the interior of the instrumented BSC 500, asshown as reference number 1002 in FIG. 10 . A mouse or touch sensitivescreen may be used in a manner well known in the art to input commandsor notes to ELN logic 300.

In some embodiments, instrumented BSC 500 may be equipped with a surfacefor accepting written notes via an electronic pencil. The surface maybe, for example, an actual contact sensitive surface sensitive tocontact with electronic pencil or may be a virtual area drawn on aninterior surface of the instrumented BSC 500 which may recognizegestures or movements of an electronic (or regular) pencil and maytranslate those movements into text by analysis of the writing gesturesvia a machine learning model trained to recognize writing gestures.

In some embodiments, instrumented BSC 500 may be equipped with anexternal foot pedal. The external foot pedal may be used to indicatepredefined actions based on an activation of the foot pedal or anactivation of the foot pedal with a certain combination or number ofpresses. Alternatively, the external foot pedal may be used inconjunction with a video or projected display to select displayedoptions or may be used in response to an audio cue. For example, ELNlogic 300 may instruct the user to “press the foot pedal to continue”via an audio cue.

In some embodiments, instrumented BSC 500 may display virtual buttons,as shown by reference number 1002 in FIG. 10 , on an internal orexternal surface of instrumented BSC 500. Such virtual buttons 1002 maybe drawn with a projector or a laser and instrumented BSC 500 may becapable of recognizing when a user has touched a particular area of theinstrumented BSC 500 on which the buttons are drawn and recognizing sucha touch as an indication of a press of the virtual button 1002. Suchvirtual buttons 1002 may indicate specific actions, or may be, forexample, a virtual keyboard on which the user can type commands.

Instrumented BSC 500 may be equipped with a processor executing BSClogic 530. In some embodiments, BSC logic 530 may collect inputs to oroutputs from instrumented BSC 500 and format them in a manner suitablefor communication to and from ELN logic 300, or format them in a mannersuitable for display within instrumented BSC 500. In addition, some ofthe functionality previously described as being provided by ELN logic300 may be implemented in BSC logic 530. For example, machine learningalgorithms trained to recognize various voice commands or gestures mayreside within BSC logic 530 instead of with ELN logic 300. In addition,BSC logic 530 may implement various security protocols or encryptionschemes for safeguarding the integrity of data exchanged betweeninstrumented BSC 500 and ELN logic 300. Such divisions of labor betweenELN 300 and BSC logic 530 are within the intended scope of theinvention.

Sensing Vessels

After an experimental protocol is set-up in the instrumented BSC 500,cultures are typically moved to an incubator where they must bemonitored over a period of time to determine the results of theexperiments. Such monitoring may be labor-intensive as each culture maybe required to be frequently examined and measured. Further, continuousexamination and measurement of the cell cultures may result incontamination.

To improve the process of examining and measuring cell cultures, and toreduce the possibility of contamination of the cultures, the connectedecosystem for the lab environment is provided with sensing vessels 1100and an associated sensing plate 1120, shown in FIG. 11 and FIG. 12 .

Sensing vessels may be of any particular shape and are illustrated inFIG. 12 as closed vessels 1100(a) or plates containing cultures 1100(b),although any shape of vessel may be used. Sensing vessels 1100 may beoutfitted with a plurality of active sensors 1110 or passive sensors1112, depending on the parameters of the experimental protocol. Readingsof measurements from both active sensors 1110 and passive sensors 1112may be sent to ELN logic 300 for analysis and storage in protocol datastorage 400. Examples of parameters that active sensors 1110 and passivesensors 1112 may be configured to detect include: pH, oxygen, lactate,glucose, temperature, location, etc. The sensors may also be configuredto detect contamination of the cell cultures. It should be noted thatthis is not an exhaustive list, sensors for detecting any otherparameters are intended to be within the scope of the invention.

In a preferred embodiment, active sensors 1110 may be equipped with anRFID tag or proximity tag capable of transmitting a measured value to areceiver, in particular sensor reader 1114. Sensor reader 1114 may thentransmit the readings collected from active sensors 1110 to eithersensor plate 1120, which would relay the readings to ELN logic 300 ordirectly to ELN logic 300.

In alternate embodiments, active sensors 1110 may be equipped withwireless transmitters, for example, Wi-Fi or Bluetooth, and may be ableto transmit their measurements to either sensing plate 1120, which mayrelay the results to ELN logic 300 as sensor inputs 212. Alternatively,active sensors 1110 equipped with wireless transmitters may transmittheir measurements directly to ELN logic 300. Examples of active sensors1110 are shown in FIG. 12 and may be specific to the parameter of theexperiment being measured. For instance, particular active sensors 1110may be able to measure oxygen, glucose, lactate, pH, temperature, etc.Various experimental protocols may call for differing configurations ofsensors within sensing vessels 1100.

In an alternate embodiment, sensing vessels 1100 may be outfitted withone or more passive sensors 1112 that may indicate measurements by, forexample, changing color or illuminating various segments of an LCDdisplay. Passive sensors 1112 must be read using a camera 1122 typicallymounted on the sensing plate 1120 upon which sensing vessels 1100 rests.Passive sensors 1112 may be monitored using a camera by sending an imageof the sensor to ELN logic 300 as camera input 202, which may determinereadings from the passive sensors 1112 by analysis of the image. Suchanalyses may be performed by machine learning models trained torecognize various patterns or colors exhibited by passive sensors 1112.

Sensing plate 1120 may also be equipped with a plurality of cameras 1122to provide direct observation of the cell cultures to determine, forexample, rates of growth of the cultures over time, cell morphology orcell confluence. In addition, cameras 1122 may also be used forholographic microscopy to visualize 3D cell structure for spheroids,organoids, etc. Such images may be sent to ELN logic for analysis andmay be analyzed using machine learning models trained to determine thedesired parameter. Results of the analysis may be stored in protocoldata store 400 by ELN logic 300. Sensing plate 1120 may also be equippedwith one or more lights 1102 as shown in FIG. 12 to illuminate sensingvessels 1100 to improve imaging of sensing vessels 1100 by cameras 1122.

Sensing plate 1120 may be loaded with multiple sensing vessels 1100 andmay be equipped with multiple cameras 1122 and multiple lights 1102.Sensing plate 1120 may have a means for transmitting the results ofmeasurements from active sensors 1110 and passive sensors 1112 as wellas images captured by cameras 1122 to ELN logic 300. This means oftransmitting may comprise a wired or wireless connection to ELN server200, such as Wi-Fi or Bluetooth.

In one embodiment, the sensing vessels 1100 may provide a continuousmonitoring of the cell cultures in an incubator and may report theresults of measurements periodically to ELN logic 300, as required bythe experimental protocol. ELN logic 300 may use the receivedmeasurements to monitor the progress of the cell cultures and to providenotifications or alarms 220 regarding progress of the cultures. FIG. 13shows an example, as reference number 1302, of a user receiving anotification on a user device 150, in this case a smart phone that tumorcells have shrunk in size. FIG. 14 shows an example, as reference number1402, of a user receiving a notification of deviations between actualsensed results and expected results as set forth during the input of theprotocol using protocol input component 310 of ELN logic 300. Protocolmonitoring component 330 of ELN logic performs the required comparisonsbetween expected conditions and results and received measurements fromsensing vessels 1100 and sensing plates 1120.

Computing Architecture

The above-described systems may be embodied as hardware accompanied by aprocessor executing instructions from a non-volatile, computer-readablemedium. A computing architecture suitable for use in support of thesystems and apparatuses is shown in FIG. 15 , which illustrates anembodiment of an exemplary computing architecture 1500 suitable forimplementing the various embodiments as previously described. In oneembodiment, the computing architecture 1500 may, in whole or in part,comprise or be implemented as part of an electronic device, such as acomputer, smartphone or tablet computing device 1550. The embodimentsare not limited in this context.

As used in this application, the terms “system” and “component” areintended to refer to a computer-related entity, either hardware, acombination of hardware and software, software, or software inexecution, examples of which are provided by the exemplary computingarchitecture 1500. For example, a component can be, but is not limitedto being, a process running on a processor, a processor, a hard diskdrive, multiple storage drives (of optical and/or magnetic storagemedium), an object, an executable, a thread of execution, a program,and/or a computer. By way of illustration, both an application runningon a server and the server can be a component. One or more componentscan reside within a process and/or thread of execution, and a componentcan be localized on one computer and/or distributed between two or morecomputers. Further, components may be communicatively coupled to eachother by various types of communications media to coordinate operations.The coordination may involve the uni-directional or bi-directionalexchange of information. For instance, the components may communicateinformation in the form of signals communicated over the communicationsmedia. The information can be implemented as signals allocated tovarious signal lines. In such allocations, each message is a signal.Further embodiments, however, may alternatively employ data messages.Such data messages may be sent across various connections. Exemplaryconnections include parallel interfaces, serial interfaces, and businterfaces.

The computing architecture 1500 includes various common computingelements, such as one or more processors, multi-core processors,co-processors, memory units, chipsets, controllers, peripherals,interfaces, oscillators, timing devices, video cards, audio cards,multimedia input/output (I/O) components, power supplies, and so forth,all of which are able to communicate as necessary using appropriateconnections. The embodiments, however, are not limited to implementationby the computing architecture 1500.

As shown in FIG. 15 , the computing architecture 1500 comprises computer1550 comprising a processor 1502, and a system memory 1504. Theprocessor 1502 can be any of various commercially available processors.Dual microprocessors, multi-core processors, and other multi-processorarchitectures may also be employed as processor 1502.

An interface is provided for system components including, but notlimited to, the system memory 1504 to the processing unit 1502. Theinterface can be any of several types of bus structure that may furtherinterconnect to a memory bus (with or without a memory controller), aperipheral bus, and a local bus using any of a variety of commerciallyavailable bus architectures. Example slot architectures may includewithout limitation Accelerated Graphics Port (AGP), Card Bus, (Extended)Industry Standard Architecture ((E)ISA), Micro Channel Architecture(MCA), NuBus, Peripheral Component Interconnect (Extended) (PCI(X)), PCIExpress, Personal Computer Memory Card International Association(PCMCIA), and the like.

The computing architecture 1500 may comprise a non-volatile,computer-readable storage medium, such as a hard disk drive (HDD) 1506or solid-state drive to store logic. Examples of a computer-readablestorage medium may include any tangible media capable of storingelectronic data, including volatile memory or non-volatile memory,removable or non-removable memory, erasable or non-erasable memory,writeable or re-writeable memory, and so forth. Examples of logic mayinclude executable computer program instructions implemented using anysuitable type of code, such as source code, compiled code, interpretedcode, executable code, static code, dynamic code, object-oriented code,visual code, and the like. Embodiments may also be at least partlyimplemented as instructions contained in or on a non-transitorycomputer-readable medium, which may be read and executed by one or moreprocessors to enable performance of the operations described herein.

The system memory 1504 may include various types of computer-readablestorage media in the form of one or more higher speed memory units, suchas read-only memory (ROM), random-access memory (RAM), dynamic RAM(DRAM), Double-Data-Rate DRAM (DDRAM), synchronous DRAM (SDRAM), staticRAM (SRAM), programmable ROM (PROM), erasable programmable ROM (EPROM),electrically erasable programmable ROM (EEPROM), flash memory, polymermemory such as ferroelectric polymer memory, ovonic memory, phase changeor ferroelectric memory, silicon-oxide-nitride-oxide-silicon (SONOS)memory, magnetic or optical cards, an array of devices such as RedundantArray of Independent Disks (RAID) drives, solid state memory devices(e.g., USB memory, solid state drives (SSD) and any other type ofstorage media suitable for storing information. A basic input/outputsystem (BIOS) can be stored in a non-volatile portion of system memory1504.

The drives and associated computer-readable media provide volatileand/or nonvolatile storage of an operating system 1520, applications1522, and related data and data structures 1524. Applications 1522 maybe in the form of software comprising computer-executable instructions.In one embodiment, the one or more applications 1522 and data 1524 maycomprise, for example, the various applications and/or components of theconnected ecosystem 100.

A user can enter commands and information into the computer 1550 throughone or more wire/wireless input devices, for example, a keyboard 1510and a pointing device, such as a mouse 1512. Other devices 1514 mayinclude both input devices and output devices as described herein insupport of the connected ecosystem 100. These may include, for example,camera inputs 202, voice inputs 206, scanner inputs 208 user inputs 210and sensor inputs 212, audio outputs 224 and video outputs 222 andnotifications outputs 220 in instrumented BSC 500, inputs from cameras1122 on sensing plates 1120 and readings from active sensors 1110 insensing vessels 1100. Other types of user input devices 1514 mayinclude, for example, microphones, infra-red (IR) remote controls,radio-frequency (RF) remote controls, game pads, electronic pencils,card readers, dongles, finger print readers, gloves, graphics tablets,joysticks, keyboards, retina readers, touch screens (e.g., capacitive,resistive, etc.), trackballs, trackpads, sensors, projectors, lasersscanners and the like. These and other input and output devices areoften connected to computer 1550 via various means, including serialports, USB connections, wired network connections, Wi-Fi connections,Bluetooth connections, etc.

A monitor 1508 or other type of display device may be used to providevideo output 222 to a user. The monitor 1508 may be internal or externalto the computer 1550. Monitor 1508 may act as both a display device andas an input device, as in the case of a touchscreen display commonlyfound on smartphones and tablet computing devices. In addition to themonitor 1508, a computer typically includes other peripheral outputdevices, such as speakers, printers, and so forth which may be used toprovide audio outputs 224.

The computer 1550 may operate in a networked environment using logicalconnections via wire and/or wireless communications to one or moreremote, networked computers, such a computer supporting instrumented BSC500. The networked computer can be a workstation, a server computer, arouter, a personal computer, portable computer, microprocessor-basedentertainment appliance, a peer device or other common network node, andtypically includes many or all of the elements described relative to thecomputer 1550. The logical connection depicted includes connectivity toa local area network (LAN) or wide area network (WAN) 110. Such LAN andWAN networking environments are commonplace in offices and companies,and facilitate enterprise-wide computer networks, such as intranets, allof which may connect to a global communications network, for example,the Internet. Computer 1550 may be connected to the LAN/WAN 110 via awired or wireless communication network interface or adaptor 1516.Network adapter 1516 can facilitate wired or wireless communications tothe LAN/WAN 110, which may also include a wireless access point disposedthereon for communicating with the wireless functionality of the networkadaptor 1516.

A connected ecosystem 100 for a laboratory environment, as well as thecomputing architecture 1500 sufficient to support the connectedecosystem 100 has been described herein. Exemplary physical and logicalcomponents and arrangements of components have been used in thedescription of the connected ecosystem 100, however, as will be realizedby one of skill in the art, many different arrangements of the physicaland logical components, or substitutions therefor, may be used withoutdeviating from the intended scope of the invention. For example, variousfunctions described as being provided by ELN logic 300 may, in someembodiments, be provided by BSC logic 530. Such alternative embodimentsare intended to be within the scope of the invention.

1. (canceled)
 2. (canceled)
 3. (canceled)
 4. (canceled)
 5. (canceled) 6.(canceled)
 7. (canceled)
 8. (canceled)
 9. (canceled)
 10. (canceled) 11.(canceled)
 12. (canceled)
 13. (canceled)
 14. (canceled)
 15. (canceled)16. (canceled)
 17. (canceled)
 18. (canceled)
 19. (canceled) 20.(canceled)
 21. A biosafety cabinet comprising: one or more outputdevices for providing information to a user of the biosafety cabinetregarding setup of experimental protocols within the biosafety cabinet,the experimental protocols being stored in an electronic lab notebookassociated with the experimental protocol; and one or more input devicesfor providing input data from the user regarding setup of theexperimental protocol, the input data to be stored in the electronic labnotebook associated with the experimental protocol.
 22. The biosafetycabinet of claim 21, further comprising: a video output device foroutputting videos and still images to be displayed on a surface of thebiosafety cabinet; an audio output device for outputting audio; a camerafor capturing videos or still images; one or more scanners; and an audioinput device for capturing audio.
 23. The biosafety cabinet of claim 22,the videos and still images comprising step-by-step instructions guidingthe user through the setup of the experimental protocol.
 24. Thebiosafety cabinet of claim 22, the videos and still images comprisingwarnings to the user that the user has deviated from parameters of thesetup of the experimental protocol.
 25. The biosafety cabinet of claim22, the scanner tracking objects inserted into and extracted from thebiosafety cabinet, videos and still images comprising an inventory ofobjects required for the setup of the experimental protocol and anacknowledgment that the required objects for setup of the experimentalprotocol are present in the biosafety cabinet.
 26. The biosafety cabinetof claim 22, the videos and still images comprising one or more virtualbuttons displayed on the surface of the biosafety cabinet, the biosafetycabinet being able to determine that the user has touched the area onthe surface of the biosafety cabinet on which one of the virtual buttonsis displayed.
 27. The biosafety cabinet of claim 22, the cameracapturing images of objects in the biosafety cabinet and actions takenby the user during setup of the experimental protocol, the images beingstored in the electronic lab notebook associated with the experimentalprotocol.
 28. The biosafety cabinet of claim 22, the audio input devicecapturing commands spoken by the user.
 29. The biosafety cabinet ofclaim 21, further comprising one or more user input devices, the one ormore user input devices being selected from a group comprising atouchscreen display, a mouse and a foot actuated switch.
 30. Thebiosafety cabinet of claim 22, further comprising: a processor; anetwork connection to a server comprising electronic lab book logic andan experimental protocol data storage containing one or more electroniclab books, each electronic lab book defining an experimental protocolincluding a list of materials and steps required to set-up theexperimental protocol; and software, for execution on the processor, thesoftware configured to interface, via the network connection, with theelectronic lab book logic to provide the functions of: receiving video,including still images, from the electronic lab book logic anddisplaying the video on the surface of the biosafety cabinet; receivingaudio from the electronic lab book logic and playing the audio via theaudio output device; and receiving commands from the user and sendingthe commands to the electronic lab book logic.
 31. The biosafety cabinetof claim 30, the received video comprising step-by-step instructions forthe user to set-up the experimental protocol.
 32. The biosafety cabinetof claim 30, the received video comprising an inventory of objectsrequired for the setup of the experimental protocol indicating which ofthe required objects have been inserted into the biosafety cabinet. 33.The biosafety cabinet of claim 30, the received video comprisingancillary information to assist the user in the setup of theexperimental protocol.
 34. The biosafety cabinet of claim 30, thereceived video comprising warnings indicating the user has deviated fromthe step-by-step instructions for setting up the experimental protocol.35. The biosafety cabinet of claim 30, the software comprising one ormore machine learning models trained to recognize one or more of voicecommands, hand or eye gestures of the user and specific users based onfacial recognition.
 36. The biosafety cabinet of claim 35, the softwarefurther configured to provide the functions of: receiving voice inputfrom the user via the audio input device; interpreting the receivedvoice input as a command; and sending the command to the electronic labnotebook logic on the electronic lab notebook server via the networkconnection.
 37. The biosafety cabinet of claim 35, the software furtherconfigured to provide the functions of: receiving video input from theuser via the video input device, the video comprising hand or eyegestures of the user; interpreting hand or eye gestures of the user inthe received video as a command; and sending the command to theelectronic lab notebook logic on the electronic lab notebook server viathe network connection.
 38. The biosafety cabinet of claim 35, thesoftware further configured to provide the functions of: receiving, viathe video input device and image containing a facial image of the user;and identifying and authenticating the user based facial imagerecognition of the user.
 39. The biosafety cabinet of claim 30, thesoftware further configured to provide the functions of: receiving videofrom the video input device; and sending the video to the electronic labnotebook logic for storing in the electronic lab notebook associatedwith the experimental protocol.
 40. The biosafety cabinet of claim 30,the software further configured to provide the functions of: receivingaudio from the audio input device, the audio comprising a note spoken bythe user; and sending the note to the electronic lab book logic forstoring in the electronic lab notebook associated with the experimentalprotocol.